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Estimating Parameters in the Rasch Model; 
Removing the Effects of Randoa Gueasing 

Abstract 

A method of estimating the parameters of the Rasch Model 
removing the effect of random guessing is presented* The pro- 
cedure is an application of the ARRG model recently developed 
for two parameter latent trait models* Under the Rasch model 
ARRG provides for estimation of abilities^ removing the effects 
of random guessing^ without requiring the use of a computer* 
Monte Carlo simulations are employed to examine the accuracy of 
the resulting ability estimates . 
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1, Introduction 

This paper presents a method of estimating the parameters of 
the Rasch model removing the effects of random guessing. The 
method used Is an application of the ARRG (Abilities Removing Randon 
Guessing) model or procedure developed recently for two parameter 
latent trait models by Waller (1973, 1974a). Application to the 
Rasch model Is more or less s t ralgh tf orward^ and the JiaX^on tt^iC 
of the present work Is to present a modified version which under 
the Rasch model becomes computer free. This Is possible because 
one characteristic which the one parameter Rasch model possesses 
and which the two and three parameter logistic latent trait models 

^ — 'do"^Trot"^l-s-^th^a"r^he"Traw^ ^core^ drs^ a— suf-f-lc-lent— 'S'ta^t-lS't-l-c---f'0'r--eS't-l»at^^^ — 

of ability. Consequen t ly » once the Item difficulties In a test have 
been estimated » a user may simp ly add up an examinee * s raw score and 
look up the examinee * s ability In a t ab le developed during estimation 
of Item difficulties. The method presented here provides the same 
facility while concomitantly removing the effects of random guessing. 

The underlying assumption of the ARRG procedure Is that examinees 
who guess In an essentially random manner^ do so on those Items , 
which are too difficult for them. Latent trait models such as Rasch 
possess two characteristics which enable us to apply this assumption 
when estimating an examlnee*s ability. Firsts these models may be 
stated In a form which enables us to obtain an estimate of the probabi- 
lity of a correct answer for each examinee's response to each Item. 
This allows us to Identify Items which are too difficult for a given 
examinee * and to identify examinees for which a given Item 1b too 

ERIC difficult . A 



Second^ once a aet of Items ±b callbrfited^ l*e*» scaled 
difficulties have been determined^ any subset of Items may be 
used to estimate an examlnee^s ability. This allows us to remove 
any number of Items In a given test and Indeed different sets for 
different examinees while still allowing, us to estimate every 
examinee on the s ame s cale CBo ck and Wood ^ 19 71) • 

Given this assumption and these two characteristics of latent 
trait models one may obtain estimates of difficulties and of 
abilities removing, random guessing by removing from the estimation 
procedure those Item-person Interactions In which a person responds 
to an Item estimated to be very difficult for him*. Sections 2 to 4 
describe the models^ calibration of the Itens^ and ability estimation 
by two methods* One method yields maximum likelihood estimates and 
requires the continual use of a computer; the other method yields 
a table from which estimates of abilities may be taken directly. 
The last section examines the accuracy of these ability estimates 
through a Monte Carlo simulation* 



-3- 



2. The Rag ch and ARRG Models 

Suppose that each of N subjects respond to M multiple choice 
items^ each Item containing alternatives^ i^l,***,H* For the 
response of person n to item i^ let: 

il denote a correct response 
0 denote an incorrect response* 



Then the probability that person n responds correctly to item i 
may be stated in Rasch model as: 

l+e n i 

where b^ is the ability of person n and d^ is the difficulty of 
item i. this probability in the ARRG model becomes: 




In (2) V ^ ts some small probability which is es tlmated during 

calibration of the test; the same value of is used for all 

subjects and items • 

Equation (2) reveals that the ARRG model divides the items 

into 2 sets: 

S * [Items such that . > P ] 
n ni — c 

S * [Items such that * ^ < P ] 
n n X c 

Only items in S are used during ability es tima tion* 
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The ARRG procedure makes the assumption that represents 
the probability below which some subjects answer randomly* While 
other sub Ject s may choose to omit 1 tems rather than guess ^ the 
ARRG procedure allows us to Ignore all responses with a low proba* 
blllty of a correct response and to do so without loss of precision 
in the resulting ability estimates (Waller, 1973 or 1974a, section 6)# 
Omitted Items in latent trait models are treated as Incorrect 
responses for estimation of ability and difficulty* (Omitted Items 
may be used as an adjunct to ARRG to obtain an Independent est Imate 
of the personality characteristic risk taking tendency as described 
In Waller (1974b). The estimate of risk taking tendency uses the 
Items not used to estimate ability, l.e*. Items In ^^*) 
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3. Test Calibration 

Estimation of the Item parameters » that Is^ calibration of 
the test, can be accomplished for the ARRG modification of the 
Rasch model by proceeding as In Waller (l974a) • However » convergence 
during test calibration Is found to be somewhat faster If an al- 
ternative procedure 1$ emp loyed • In the procedure used In this 
paper we ad J us t the responses prior to cal lb ration to ref lec t the 
ARRG model and proceed with a free response analysis as In Wright 
andPanchapakesan(X969)^. 

The required adjustment Is obtained as follows: A preliminary 
estimate of each subJect^s ability and each ltem*s difficulty Is 
obtained from a transformation of the raw proportion correct by 
the Inverse of equation (1)1 l+e** 



n 



b° » In / * n \ , P - raw proportion correct; 



(3) 



" Inl " 1 \ » " difficulty. 



Then for each 1 and n^ tp^^ Is estimated using (l). The response 

value to be used during ARRG calibration^ say a » Is obtained from 

n 1 



■I 



(4) a* - f 

— * , < P 

nl c 



1. 



ERXC author wishes to thank Benjamin Wright for suggesting this 

method of calibration.. 
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As described in Wright and Panchapakesan (196^) an estimate 
of ab ili ty is produced for each raw score as apart of the f ree 
response calibration process* In the ARRG .procedure these estimates 

if 

based on. the adjusted responses a^^ , are used only as star ting. 

points for the estimation of each subject's ability as described 

In the next section* 

The proper value for is obtained as follows; A cutoff 

probability* P^, is chosen at the beginning "of a calibration run 

2 

and remains constant during t-he run. A x goodness ,of fit statistic 

(see e.g* Wright arid Panchapakesan ^ 1969) is calculated for the test 

during each run. The value of P is then incremented and a second 

c 

calibration is per^^formed. After each calibration run* the fit if5 

examined* The x statistic is £ound t^o decrease* as is increased 

to some minimum level and then increase (Mailer * 19 74a) • The cali- 

2 

bration run with the minimum X identifies the value to be used for 

during estimation of ability, and the item difficulties obtained 

during this run become the estimated item difficulties of the test, 

2 

Note that if the minimum X calibration occurs with V ^ equal to 
zero this is an indication that the data are essentially free from 
random guessing, and in this case the resulting parameter estimates 
are identical to those produced by a standard Rasch calibration. 
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4. AblllCy EsClmacloii 

t'i Af Cer Che Items are calibrated^ est Ima clon: of ability under 

Che ARRG procedure can be performed In either of Cyo ways. The 
first method^ here labelled MAX.^ produces maximum likelihood 
edClmaced (MLEd) and requires the use of a computer.* The second 
method* here labeled TABLE^ produces, .ap^proxlmace MLEs but after 
cal lb ration does not requlrea computer • 

The first method yields maximum likelihood estimates 
by means of Newton Rapheson Iteration. This method Is Identical 
to that used In free response Rasch estimation (Wright and Fanchap.akesan 
1969) with the exception that a subset of Items reflecting the ARRG 

^M!QA^1^X^Jj^ IJ?is^,s„aj;iXAm,pJJ.:sJi^ud--^ _^ 
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Given the Initial estimate of ability from (3)» the set of 

Item difficulties » and the value of determined during calibration » 

t h 

the estimate of ability for person n at the k Iteration Is given 
In the Newton Rapheson approach by: 



(5) b^*^^ - b^*^"^^ - IJL^. 

n n D D D 



Here and t^^ are che first and second partial derivatives with 
respect to b of the log likelihood func tlon » l.e.» 



(6) 



n n 



In (6) (p^^ Is evaluated using the (k-1)^^ estimate of b^; the 

ARRG model Is reflected. In the fact that the set S consists of only 

n - 

those Iteias for which ^ P^, JQ 



The second method^ TABLE^ consists of developing a table 

utilizing the estimated abilities^, difficulties and from the 
2 

minimum x calibration run. For each raw score the table contains 

the estimated ability for that score together with a list of the 

Items for which ^ * (as calculated using the ability associated ^; 

ni 

with that score) Is greater than the cutoff probability » P^. An 

examiner obtains the estimated ability by adding up the subJect^s 

raw score,^ finding the subset of Items In the table associated with 

that raw score^ and then calculating an adjusted raw score based only 

on the Items in this subset. The subject^s estimated ability Isthe 

ability associated with this ad j us ted raw score:. 

For example ^ consider the two simulated response vectors^ Vj^ 

and taken from the Monte Carlo simulation described later. Items 

In these vectors range from easy Items on the left to difficult Items 

on the right. Both vectors yield a raw score of 14 correct out of 

45. 

V, "{101100001100100000000000010101010011010010000} 



v^"{ l 11111110 iiiiiooooooooioo ooooodoooodooooooooo} 

Tftble^ 1 presents the estimated Rasch abilities from the minimum 
Insert Table 1 about here 

calibration run on this set of simulated data^ and the subset of 
Items for which the estimated probability of a correct response Is 
greater than the cutoff point associated with this run^ ■ 0.12. ^ 
We see* that for a raw score of 14 this subset contains Items one to 
twenty- five (underlined above); and consequently ^ the subset containing 
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these Items will be used to obtain these simulated examinees ' 
estimated ARRG ability « The "examlnee^^ corresponding to the first 
response vector achieved six correct responses In this subset and 
his estimated ARRG ability Is the estimated Rasch ability corresponding 
to a raw score of slx» 9^ » -3+577+ The second "examinee" achieved 
all fourteen correct responses In this subset and his es tlma ted 
ability Is the same as the Rasch ability corresponding to his adjusted 
rawscoreoffourteen^92 = -l*:677 + 

While the disparity between the estimated abilities might seem 
unwarranted (given that both "examinees" achieved the same raw score)^ 
the difference reflects the fact that the first response vector 
simulates someojie of low ability who guesses In a more or less random 
manner at Items which he doesn't know » while the second vector 
simulates someone b( a higher ability but someone who omits items 
he doesn't know» omit ted Items being scored as Incorrect + The 
estimated standard error of these two ability estimates are * 0+538 
and O2 * 0 + 45?, Since the true ability^ l + e+ » those used to gene ra te 
these data, are 9^^ » -3*41 and 92 * -1*58, lt*s clear that 95% con- 
fidence In terval s placed around the es tlma ted abilities cover the 
true values . 

During ability estimation with either method, the original 

responses of each examinee , a ^ not a*^, are used* An examinee * s 

nl nl 

response Is never changed from right to wrong when estimating his 
ability; only the number of Items used to estimate anexaminee^s ability 
may be changed* As Indicated In the Introduction, latent trait 
models ariow estimation of abilities on the same: scale using any 
subset of calibrated Items, This characteristic may be seen in equation 

12 
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(6) of the MAX method: responses to items not In the set used to 
estimate ability^ S , do not affect estimation in any way. 
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5. A Monte Carlo Simulation 
I Evaluation of ARRG ability estimation m%y be made through 

a Monte Carlo simulation. A saaple of abilities of slse 480 from 
a logistic distribution was generated simulating both guessing and 
non-guessing subjects. The simulated test was constructed to have 
45 Items ranging In difficulty from -4.4 to +4.4 In steps of .2. 
A no n- guessing response vector was generated by setting the value 
of his ability parameter^ calculating the true probability of a 
correct response^ ^nl* each Item and then comparing this value 

to a random number^ between zero and one: l.e.» 

1 if * , > r , 
nl nl 

a , » 

nl 

0 otherwise • 

A guess In g subject*s response vector was generated In the same 
manner with the exception that whenever the calculated probability^ 
^^^t less than the random number was used to yield a 

simulated correct response on one-f If th of such Items (thus slmulat" 
Ing a five choice test) . The sample contained approximately 33X 
guessers and the remainder non-guessers . 

The simulated test thus generated was calibrated using the 
ARRG model and the free response model. As Indicated earlier^ If 
the best fitting calibration run occurs with " 0. ARRG estimates 
are Identical to free response estimates; that ls» the free response 
analysis Is subsumed by the ARRG analysis. 
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The comparison of the ARRG estimates to f ree^ response estimates 

Is Included to highlight the effect of Ignoring random guessing 

ifhen It Is present In the form modeled by the ARRG procedure* 

2 

When calibrating a real test» X statistic will Indicate 

whether anything will be gained by employing a cutoff point 
greater than zero* 

The ability for each subject was estimated by the two methods 
presented for the ARRG model and by the free response model* The 
samples were then split Into two sub samples » one composed of the 
B Imulate d guessers^ the o the r » s Imula ted non^guessers * Table 2 



Insert Table 2 about here 



presents functions of the first four moments of the samples of the 
estimated abilities* These are compared to the corresponding 
moments of the sample of true abilities (also shown) ; all slgnl^ 
f leant differences are Indicated* 

For each sub j ect an es t Ima te of the asymp to 1 1 c variance of 
the subject's estimated ability may be calculated as 

(7) a| - -1/ I- . ^^J^iKl^-^-Kl^U 

S 

where the sununatlon Is again Saken over only those Items for which 
^nl — * ^B^^i O) 95Z confidence Intervals may be placed 
around each subject's estimated ability* Table 3 presents the 
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Insert Table 3 about here 

number of tlmes^ In each group of simulated subjects » such confidence 
Intervals fall to cover the true ability* 

15 
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Results from Tables 2 and 3 Indicate that only the free raaponae 
analysis of data contaminated by guessing falls to adequately 
e 8 tlmate the true ab 111 ties • As would ba expected from a free 
response analysis of data contaminated by guesalng, the mean 
of the simulated guessera Is significantly overestimated relative 
to the true mean* Also^ this aubsample Is- seen to contaln^ a 
positive » though not satlstlcally significant » skew relative to 
the true sample of abilities^ as well as a significantly smaller 
variance. Both ARRG methods produce samples of estimated 
abilities which are statistically equivalent to the sample of 
true ab llltles . 

While the TABLE method does not produce eitact maximum like- 
lihood es tlmate s of ability (except for those sub J ects whose 
ability estimate Is based on the entire set of Items) » the ability 
estimates produced by this me thod In con J unction with their 
estimated errors are found to produce an accurate estimate of an 
examinee's ability. The TABLE method also possesses the charact- 
eristic sought In this work that after calibration of an instrument 
by a test publlsheit a u«er Is not required to possess high speed 
electronic computing facilities to obtain accurate estimates of 
ability essentially uncontamlnated by random guessing. 
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ZZ Z) 
ZZ Z) 
ZJ z) 
ZZ Zl 
11 Zl 
ZZ z) 
ZZ Zl 
17 7\ 
17 7i 



Z« Z4 

Ik Z4 Z4 Zl 
Z4 Z4 Z4 11 
Ik Z4 Z« ZT Z4 
Z4 -Z4 Z4 IJ Zt Z« 
Z«.Z4 Z4 Zl Z4 Z4 )0 
Z« Z4 Z4 ZT Z4 Z« 10 11 
Z« Z4 Z4 IJ Z4 Z« )0 11 
Z« Z4 Z6 Zl Z4 Z4 10 11 
7ft Z4 Z4 Zf Z4 Z« 10 11 
/4 Z4 ZT Z4 Z« 10 II 
A Z4 Z4 Zf Z4 Z4 10 )1 



4 lb 14 If 14 ZO ^1 71 Zl Ik Z4 Z4 Zf Z4 Z« )0 11 

) 16 14 If M ZO Zl Z^ Z) Ik Z4 Z4 Zl Z4 Z4 10 )1 

) U Ii II 1^ ZO Zl ZZ Zl Z« Z4 Z4 Zf Z4 Z« )0 11 

4 16 |4 If |4 ZO Zl ZZ Zl Z^ Z4 Z6 Zl Z4 Z« 10 )1 

) U 14 iJ M ZO Zl ZZ Z) Zt Z4 Z4 Zf Z4 19 )0 11 

) ii> 14 If ll ZO Zl /Z Z) Zt Z4 Z4 Zf Z4 Z« 10 11 

) lb 14 If 1^ ZO Zl 77 Z) ZV Z4 Z4 Zl Z4 Z« tO 11 

) U 14 If 19 ZO Zl 77 Z) ^t Z4 Z4 17 Z4 Z« 10 11 

) 16 14 If I'J ZO Zl 17 Zt Zt Z4 Z4 Zf Z4 Z« 10 11 

V 16 la ir 14 zo Zl ZZ ZI Z4 Z4 Z6 Zl Z4 19 10 11 

^ lb Ii If n zo Zl ZZ z) Zt Z4 Z4 zr Z4 z« lo ii 

16 14 If VI ,10 Zl ZZ Z) ^t Z4 Z6 Zf Z4 Z« 10 11 



^ ;i0 4 ^IIMIZ It't^' Ih I A If n /O Zl /Z Zl Z4 14 Z4 Zf Zl Z4 10 11 

U 4Z I'J 41 41 ^4 44 

16 14 If X9 ZO Zl ZZ Z) Zt Z4 Z4 Zl Z4 Z« 10 11 



16 14 it Vf 10 Zl 17 Z) Zt Z4 Z6 Zl Z4 Z4 10 11 

l6 M IZ I* ZO Zl ZZ Zt Z4 Z) Z4 ZZ Z4 Z« )0 it 

lb 14 IZ t'l ZO Zl ZZ Zl Zt Z4 Z4 Zl Z4 19 10 11 

lb 14 If 14 ZO Zl ZZ Zl Zt Z4 Z4 Zl Z4 19 K 11 

16 14 If 1/ ZO Zl 12 Zl ;v Z4 Z4 Zf Z4 29 K 11 



I) 

I) IZ 

I) IZ 1^ 14 
11 IZ It 14 
11 IZ It 14 

11 IZ 14 

11 IZ It 14 

11 IZ It J4 

11 IZ It 1) 

11 IZ 16 14 

11 IZ It 14 

11 IZ It 14 

11 IZ It 14 

II IZ It 14 

11 IZ It 14 

11 IZ It 14 

11 IZ 14 

11 iz -J^ 14 
11 IZ It 14 
11 IZ ]t 14 
II IZ 16 14 
11 IZ It 14 
11 IZ 16 14 
11 iZ 14 14 



I 
10 
IZ 
11 
It 
14 
14 
19 
Zl 
ZZ 
ZZ 
ZZ 
Z4 
Zl 
Zl 
It 
19 
10 
11 
IZ 
1* 
1* 
14 

14 

ll 

1* 



t| 
tl 
tl 
t4 
t4 
t4 
t4 
t4 
t4 
t4 
t4 
t4 
t4 
t4 



18 



64 
t« 
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Table 2 

Moments of the Recovered Samples of Abilities 





Analysis 


Mean 


Variance 


Sk«wness 


Kur cotlt 


Free 


Response 
Guessers 
Hon Guessers 


0,1211^ 
0 . 3095 
0.0261 


2.142** 
1 . I'l^ 
2.238" 


0.1030 
0.6545 
-0.0849 


0.6794 
0.3196 


ARRG- 


-MAX 

Guessers 
Non Guessers 


-0.0171 
-0.0247 
-0.0133 


2.301 
3.032 
2.693 


0.1721 
0.4938 
0.0184 


0.4482 
0.6?72 
0. 3242 


ARRG^ 


-TABLE 
Guessers 
Hon Guessers 


-0.0887 
-0.0716 
-0.0974 


2.916 
3. 123 
2.824 


0.1758 
0.4903 
0.0075 


0.4472 
0.6386 
0,3369 


TRUE 


Guessers 
Mon Guessers 


-0.0770 
-0.0407 
-0.0604 


2.769 
2.957 
2.692 


0.1498 
0.3001 
0.2414 


0.4674 
0.7382 
0.1369 



Fit X 
D.F.-135 
267.81 



l€4.n& 



164.08 



a 




.009 


b 


P << 


.001 


c 


f < 


.001 



Sample Size 

Full Sample 477 
Guessers 160 
Ffon Guessers 377 
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Table 3 

Nunbcr of Estimated Confidence Intervels 
Felling to Cover the True Ability 

i 

Seaple Number Felling P 



Anelysls N To Cover Ability > Less Tben 

Free Response 477 59 .0001 

Guessers 160 37 .0001 

Non Guessers 377 22 .0776 

ARRG MAX 477 21 .9542 

Guessers 160 5 .9161 

Non Guessers 377 16 .5205 

ARRG TABLE 47 7 24 .5540 

Guessers 160 8 .5506 

Non Guessers 377 16 .5205 
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